The All-pairs Pipeline

Successful exploitation of parallel computers depends to a large extent on the development of useful concepts which enable programmers to view different applications as variations of a common theme. Our most fundamental concepts, such as parallel processes and message communication, are embedded in programming languages. In other cases, we discover programming paradigms which can be used to solve a class of applications. An all-pairs problem is a computation on every possible subset consisting of two elements chosen from a set of n elements. N -body simulation is an all-pairs problem (Fox 1988, Brinch Hansen 1991). Householder reduction of a matrix to triangular form is a less obvious example (Press 1989, Brinch Hansen 1990a). This paper develops the all-pairs paradigm discussed by Shih (1987) and Cosnard (1988). I define the problem concisely by means of precedence matrices and derive a parallel algorithm. The algorithm is presented in both coarse-grain and medium-grain form. The all-pairs paradigm is illustrated by a pipeline for Householder reduction. Pipeline algorithms for matrix reduction have already been developed based on a detailed understanding of various reduction methods, such as Gaussian elimination, Givens rotations, and Householder reduction (Ortega 1988).